1. Field of the Invention
The present invention generally relates to the field of digital image processing. More particularly, the invention relates to providing a system and method for increasing the effective dynamic range of image sensors by combining information from multiple images of the same scene taken at varying exposures via digital image processing.
2. Description of the Related Technology
Many applications exist that require analyzing images to extract information. For example, a mobile robot that uses vision (one form of “machine vision”, which is described in greater detail below) will take and analyze a series of digital images looking for objects, both landmarks and obstacles, in order to plan its movements. Both computers and humans analyze images from security systems to assure that everything is in order. Images from satellites are analyzed by looking for objects or patterns or by comparing similar images taken at different times. Typically, greater contrast (i.e., dynamic range) in an image increases the information that can be extracted from the image because edges and other details are better defined. In machine vision, this higher contrast enables simplified edge detection and object localization.
The human vision system has a wide dynamic range and is very good at extracting information from a scene that includes both very bright and very dark areas. However, high contrast often overwhelms the dynamic range of other image sensors, for example digital or analog cameras, such that substantial areas are either saturated or very dark. For a digital image, saturated areas are those that contain pixels at or near the maximum brightness, while dark areas are those that contain pixels at or near the minimum brightness. Typically, saturated areas result from overexposure, while dark areas result from underexposure. It is generally very difficult to effectively analyze saturated and dark areas. High cost cameras typically have a greater dynamic range, but are prohibitively expensive for many applications.
Exposure control is one method to adjust contrast in an image. Current auto exposure systems are generally very accurate. Typical auto exposure systems use reflected-light exposure meters within the camera to detect the level of light and electronically adjust the camera's shutter speed and/or aperture to the optimal exposure setting. Photocells are typically used as exposure meters, with the silicon photo-diode being the most common. Image sensors are typically their own exposure meters.
While these light averaging exposure control systems do a good job of selecting the correct nominal exposure, many images still contain overexposed (saturated) and underexposed (dark) areas. For example, in a photographic image that includes a light, the majority of the image may appear correctly exposed, but the area near the light will often be overexposed (saturated), appearing bright white and without detail.
Some of the digital cameras presently on the market have enhanced auto exposure systems. One such system allows the user to automatically take the same shot at three exposures with one press of the trigger (referred to as bracketing). The user can then choose the best-exposed shot of the three. These systems also allow the user to select the range between the three exposures. However, local bright and dark areas within the image may still make it impossible for any single exposure to effectively capture the entire image. Typically, photographers have used external lights or flash bulbs to improve the picture contrast by increasing the dynamic range of the scene.
Machine vision may refer to the analysis by a system (i.e., machine) of an image or sequence of images to produce useful descriptions of what is imaged in order to accomplish a given task. For example, machine vision may include extracting facial characteristics from an image in order to match a person in an image with a database of facial characteristics, or determining the distance to an object from two stereo images in order for a robot to determine how far it can move forward without hitting the object. Following are examples of applications that may incorporate machine vision processing:                1. Determining a map of an environment in order for a robot to move about autonomously without running into objects;        2. Identifying people in an image to determine who is at a certain location;        3. Tracking an object across multiple images to determine where the object is in the images (e.g., determining where a soccer ball is in the images so that it can be replaced with an advertisement in the shape of the soccer ball in the images);        4. Determining the location, orientation, and size of an object so that it can be automatically manipulated by a machine (e.g., a robotic arm needs to pick up an object from an assembly line, and the object can be one of many types and different places/orientations on a conveyor belt);        5. Tracking an object across multiple images to determine where it is in the environment (e.g., what are the visual characteristics, location, speed, and trajectory of an incoming object to determine whether it is a missile or a plane); and        6. Analyzing medical images in order to find likely areas of disease.        
In machine vision applications, one example being robotics, it is advantageous to have the ability to view a whole scene with all areas having sufficient contrast to recognize and analyze the features contained therein. In vision robotics, the robots navigate through the environment by recognizing objects, open space, patterns, walls, and the like. One implementation of robotic vision employs two cameras for stereo vision to determine the distance to objects. Poorly exposed images that lack sufficient dynamic range can lead to the robots either getting lost or failing to adequately locate and avoid objects or dangers such as an open stairwell. Therefore, there is a need in the vision robotics and image sensing and processing technology for increasing the effective dynamic range of image sensors. Such an enhancement in effective dynamic range would be useful for numerous applications in addition to the robotic application mentioned above.